{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pima Indians Diabetes Data Set——Logistic回归\n",
    "数据说明： Pima Indians Diabetes Data Set（皮马印第安人糖尿病数据集） 根据现有的医疗信息预测5年内皮马印第安人糖尿病发作的概率。\n",
    "\n",
    "数据集共9个字段: 0列为pregnants(怀孕次数)； 1列为Plasma_glucose_concentration(口服葡萄糖耐量试验中2小时后的血浆葡萄糖浓度)； 2列为blood_pressure(舒张压,单位:mm Hg） 3列为Triceps_skin_fold_thickness(三头肌皮褶厚度,单位：mm） 4列为serum_insulin(餐后血清胰岛素,单位:mm） 5列为BMI,体重指数（体重（公斤）/ 身高（米）^2） 6列为Diabetes_pedigree_function(糖尿病家系作用) 7列为Age(年龄) 8列为Target(分类变量,0或1）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#导入必要的工具包\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pregnants_log</th>\n",
       "      <th>Plasma_glucose_concentration_log</th>\n",
       "      <th>blood_pressure_log</th>\n",
       "      <th>Triceps_skin_fold_thickness_log</th>\n",
       "      <th>serum_insulin_log</th>\n",
       "      <th>BMI_log</th>\n",
       "      <th>Diabetes_pedigree_function_log</th>\n",
       "      <th>Age_log</th>\n",
       "      <th>Target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.673239</td>\n",
       "      <td>0.944441</td>\n",
       "      <td>0.891583</td>\n",
       "      <td>0.778151</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.839581</td>\n",
       "      <td>0.356534</td>\n",
       "      <td>0.639050</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.239812</td>\n",
       "      <td>0.840710</td>\n",
       "      <td>0.873760</td>\n",
       "      <td>0.738561</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.786030</td>\n",
       "      <td>0.195523</td>\n",
       "      <td>0.284791</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.760188</td>\n",
       "      <td>0.984263</td>\n",
       "      <td>0.867462</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.755862</td>\n",
       "      <td>0.380165</td>\n",
       "      <td>0.308180</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.239812</td>\n",
       "      <td>0.849290</td>\n",
       "      <td>0.873760</td>\n",
       "      <td>0.690106</td>\n",
       "      <td>0.675479</td>\n",
       "      <td>0.798568</td>\n",
       "      <td>0.068711</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.929966</td>\n",
       "      <td>0.771702</td>\n",
       "      <td>0.778151</td>\n",
       "      <td>0.760921</td>\n",
       "      <td>0.897058</td>\n",
       "      <td>0.965907</td>\n",
       "      <td>0.330870</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   pregnants_log  Plasma_glucose_concentration_log  blood_pressure_log  \\\n",
       "0       0.673239                          0.944441            0.891583   \n",
       "1       0.239812                          0.840710            0.873760   \n",
       "2       0.760188                          0.984263            0.867462   \n",
       "3       0.239812                          0.849290            0.873760   \n",
       "4       0.000000                          0.929966            0.771702   \n",
       "\n",
       "   Triceps_skin_fold_thickness_log  serum_insulin_log   BMI_log  \\\n",
       "0                         0.778151           0.000000  0.839581   \n",
       "1                         0.738561           0.000000  0.786030   \n",
       "2                         0.000000           0.000000  0.755862   \n",
       "3                         0.690106           0.675479  0.798568   \n",
       "4                         0.778151           0.760921  0.897058   \n",
       "\n",
       "   Diabetes_pedigree_function_log   Age_log  Target  \n",
       "0                        0.356534  0.639050       1  \n",
       "1                        0.195523  0.284791       0  \n",
       "2                        0.380165  0.308180       1  \n",
       "3                        0.068711  0.000000       0  \n",
       "4                        0.965907  0.330870       1  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train = pd.read_csv('FE_diabetes_log.csv')\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 768 entries, 0 to 767\n",
      "Data columns (total 9 columns):\n",
      "pregnants_log                       768 non-null float64\n",
      "Plasma_glucose_concentration_log    768 non-null float64\n",
      "blood_pressure_log                  768 non-null float64\n",
      "Triceps_skin_fold_thickness_log     768 non-null float64\n",
      "serum_insulin_log                   768 non-null float64\n",
      "BMI_log                             768 non-null float64\n",
      "Diabetes_pedigree_function_log      768 non-null float64\n",
      "Age_log                             768 non-null float64\n",
      "Target                              768 non-null int64\n",
      "dtypes: float64(8), int64(1)\n",
      "memory usage: 54.1 KB\n"
     ]
    }
   ],
   "source": [
    "train.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据准备 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train = train['Target']\n",
    "X_train = train.drop(['Target'], axis = 1)\n",
    "\n",
    "#保存特征名字以备后用（可视化）\n",
    "feat_names = X_train.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 默认参数的Logistics Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "lr = LogisticRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "logloss of each fold is: [0.56416188 0.57478428 0.61565209 0.56976182 0.58162175]\n",
      "cv logloss is: 0.581196363512364\n"
     ]
    }
   ],
   "source": [
    "#交叉验证用于评估模型性能和进行参数调优（模型选择）\n",
    "#采样5折交叉验证\n",
    "from sklearn.model_selection import cross_val_score\n",
    "loss = cross_val_score(lr, X_train, y_train, cv = 5, scoring = 'neg_log_loss')\n",
    "print('logloss of each fold is:', -loss)\n",
    "print('cv logloss is:', -loss.mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "结果比原始特征（0.47615970944434044）差？？？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 正则化的 Logistic Regression及参数调优\n",
    "logistic回归的需要调整超参数有：C（正则系数，一般在log域（取log后的值）均匀设置候选参数）和正则函数penalty（L2/L1） 目标函数为：J = C sum(logloss(f(xi), yi)) + penalty"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise',\n",
       "       estimator=LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
       "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
       "          verbose=0, warm_start=False),\n",
       "       fit_params=None, iid=True, n_jobs=4,\n",
       "       param_grid={'penalty': ['l1', 'l2'], 'C': [0.0001, 0.001, 0.1, 1, 10, 100, 1000]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring='neg_log_loss', verbose=0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#导入模块\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "#设置参数搜索范围\n",
    "penaltys = ['l1', 'l2']\n",
    "Cs = [0.0001, 0.001, 0.1, 1, 10, 100, 1000]\n",
    "tuned_parameters = dict(penalty = penaltys, C = Cs)\n",
    "\n",
    "#生成学习器实例\n",
    "lr_penalty = LogisticRegression(solver = 'liblinear')\n",
    "\n",
    "#生成GridSearchCV的实例\n",
    "grid = GridSearchCV(lr_penalty, tuned_parameters, cv = 5, scoring = 'neg_log_loss', n_jobs = 4)\n",
    "\n",
    "#调用GridSearchCV的fit方法\n",
    "grid.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.5380762876709355\n",
      "{'C': 100, 'penalty': 'l2'}\n"
     ]
    }
   ],
   "source": [
    "# examine the best model\n",
    "print(-grid.best_score_)\n",
    "print(grid.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "d:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "d:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xl8TNf7wPHPmeyJbLJKQkJQYgliqe2LoqWKbrFTWqVqrVZRutBStRRtVe37rvWroNRO7bSxxNJaEmIJErElke38/phJTEhkEjOZkPN+veaVmXvPvfcZ2jzOfc49R0gpURRFUZQn0Zg7AEVRFKXwU8lCURRFyZVKFoqiKEquVLJQFEVRcqWShaIoipIrlSwURVGUXKlkoSiKouRKJQtFURQlVypZKIqiKLmyNHcAxuLu7i4DAgLMHYaiKMoz5ciRIzellB65tXtukkVAQACHDx82dxiKoijPFCFElCHt1G0oRVEUJVcqWSiKoii5UslCURRFydVzU7NQFEXJkJKSQnR0NElJSeYOpdCwtbXFz88PKyurfB2vkoWiKM+d6OhoHB0dCQgIQAhh7nDMTkpJbGws0dHRlC5dOl/nULehFEV57iQlJeHm5qYShY4QAjc3t6fqaalkoSjKc0kliqye9s9DJQtFURSg/Yx9tJ+xz9xhFFoqWaD+I1EUxfiKFSuW+b5Fixa4uLjw2muvZW574403qFatGmXLlsXZ2Zlq1apRrVo19u7dm6frbNu2jf379xst7pyoAjfwRewQ3bu/zBqHoijPpyFDhpCQkMCMGTMyt61ZswaAHTt2MHHiRNatW5evc2/btg13d3defPFFo8SaE9WzUBRFMbGmTZvi6OhocPtDhw7RqFEjQkJCaNmyJTExMQBMnjyZoKAggoOD6dKlC+fOnWP27NlMmDAhX72SvFA9C0VRnmujwiI4eeVOru1OXtW2MeSWdJCPE1+2rvTUsWXnwYMHDBw4kLVr1+Lu7s6SJUv4/PPPmTlzJuPHjycqKgpra2vi4+NxcXGhZ8+euLu7M2jQIJPEk6HIJ4uklDSWPmhAM6tjmOavXlEUxXCnTp0iIiKCZs2aAZCWloafnx8AlSpVokuXLrRt25bXX3+9QOMq8ski7n4yqx/U4UKaB03MHYyiKEZnaA8go0exonddU4aTKyklVatWZffu3Y/t27RpEzt37uT333/nm2++4cSJEwUWV5GvWfi42NHRZg97U19gx5nr5g5HUZQiLigoiMuXL3Pw4EEAkpOTiYiIIC0tjejoaF566SUmTJjAjRs3SEhIwNHRkbt375o8riKfLLgVxVCxiLctdvPV2ggepKaZOyJFUZ4zDRs2JDQ0lK1bt+Ln58emTZtybGtjY8Pq1asZPHgwwcHBVK9enQMHDpCamkqnTp2oWrUqNWrUYOjQoTg6OtK2bVtWrlxJ9erVVYHbpBxLYEEaw62WERLbkNm7L9C3SVlzR6UoyjPu3r17me+zu6WUoXHjxjRu3DjLtho1avDXX48P5d+zZ89j2ypUqMDx48fzH6iBVLKwtOaGhSe+aZcZGnCeqdssaFvNBz9Xe3NHpihKATJ3raKwU7ehAF/fALC05b3U5WiQfL3upLlDUhRFKVRUsgAQApxLYX0zgu+rXGRTRIwqdiuKouhRySKDgwe4l+fl6/Mo62arit2Koih6VLLIIAQ0Hobm5ml+DI4iMjaBWbvOmzsqRVGUQkElC31Bb4BnEBXPTOO1yh78tP0s0bcSzB2VoigFYV4r7UvJlkoWAD3Wa18aDTQeDrFn+brMKQRCFbsVRcmXgpiifNq0aSxZssSocefEpMlCCNFCCHFGCHFWCDEshzbthBAnhRARQoilum3VhBD7dNuOCSHamzLOHht70GNjD+2Hiq3BuyquhyYzsIm/KnYrivLUhgwZwqJFi7JsW7NmDeHh4cyePZuGDRsSHh5OeHg49erVy9IuNTU1x/P27duXzp07myTmR5ksWQghLIBpQEsgCOgohAh6pE05YDhQX0pZCciYNjEB6Kbb1gKYIoRwMVWsjwQOTT6DWxfo6XiQMh4OqtitKMpTyesU5X5+fnz99dfUr1+fNWvW8Msvv1CrVi2Cg4MJDQ0lMTERgJEjRzJlyhQAGjRowLBhw6hduzYvvPCC0Z/mNuVDebWBs1LK8wBCiOVAW0D/vs77wDQp5S0AKeV13c9/MxpIKa8IIa4DHkC8CeN9qHwL8KmB5V8TGd1qI13mhzNr13n6vVSuQC6vKIoR/TEMrhnwhPO1Y9qfhtQtvKtAy3FPF1cuHBwcMp/Yjo2N5YMPPgBg2LBhzJ8/nz59+jx2jJSSgwcPsnbtWkaPHs3GjRuNFo8pb0P5Apf0PkfrtukrD5QXQuwRQuwXQrR49CRCiNqANXAum329hBCHhRCHb9y4YbzIhYAmI+D2RRrc3cirVbxVsVtRlALVvv3Du+/Hjh2jYcOGVKlSheXLlxMREZHtMW+++SYAISEhREZGGjUeU/YsRDbbZDbXLwc0BvyA3UKIylLKeAAhRAlgEfCOlDL9sZNJOROYCVCzZs1Hz/10yjaFknVg9yRGdtvD9tM3+HrdSWZ0rWnUyyiKYmKG9gAyehQ91psuljxwcHDIfN+tWzf++OMPKleuzOzZs3Ncc9vGxgYACwuLJ9Y68sOUPYtooKTeZz/gSjZtfpdSpkgpLwBn0CYPhBBOwHpgpJTS9KuRPyqjdnHnMj7nVtK/aVlV7FYUxSzu37+Pt7c3KSkpLF261CwxmDJZHALKCSFKCyGsgQ7A2kfa/B9o1xwSQrijvS11Xtd+DbBQSrnKhDE+WelG4N8Adk+iZ50SqtitKEq+5GWK8uyMHj2a2rVr07x5c4KCgnI/wASElMa9e5Pl5EK8CkwBLIC5UsoxQojRwGEp5VohhAAmoR3xlAaMkVIuF0J0AeYB+jfmukspw3O6Vs2aNeXhw4fzFWfGsNl5LeY9vjNyD8x/FV4ew26P9nSdc5BPXi6vit2KUoidOnWKihUr5u2gQnYbyhSy+3MRQhyRUuZ6f92kU5RLKTcAGx7Z9oXeewkM1r302ywGFpsyNr1rceXeFdzt3LNvEFAfyjSGvybTcFCPzGL369V91TTmivI8eY6ThDEU+Se4I+9Ecu3+NU7GnmR3dA4LlDQZAQk34eBMRrYKUk92K4pS5BT5ZFHauTQV3SpiZWHFh1s/ZNLhSaSkpWRtVLI2lG0Oe6biY5uSWezerordiqIUEUU+WQD0mH6eL1ZI2r/QnvkR83ln4ztcunspa6MmwyHxFhyYQc8GZSjj4cAoVexWFKWIUMlCxypNMPLFkUxuPJnIO5G0C2vHxki9px99Q+CFV2Hfj1in3GFUm0pqGnNFUYoMlSwe0cy/GataryLQJZAhO4fw1d6vSEzVzsNC4+GQdBv2/0zDch6Zxe5LcerJbkV51mWZUFR5TJFPFjI1FdcbSVgmP7yd5FvMl3kt5tGzSk9+++83Oq3vxNlbZ6FEVajYBvb9DAlxqtitKEqOMqYoDw8Pp27dulSqVImqVauyYsUKwDhTlANs27Ytxye6janIJ4uUy5exv5eK9+UE7v21J3O7lcaKgTUG8kvzX4hLiqPj+o6s/nc1stEwSL4He3/Ex8WOAU3L8edJVexWFCV79vb2LFy4kIiICDZu3MigQYOIj483eIry3KhkUUCs/f255mdPqqWGS717E7doMfoPKtbzqcevbX6lumd1Ru0bxadn5nO3Uhs4MAPu3+S9BqUzn+xOSlHFbkVRsipfvjzlymkf4vXx8cHT05PcJj49dOgQjRo1IiQkhJYtWxITEwPA5MmTCQoKIjg4mC5dunDu3Dlmz57NhAkT8tUryQuTPpT3rEiz0hDja0/F0rWJGTOGB2fP4j1yBMLKCgB3O3d+af4Lc0/M5ad/fuK4rRsTNKlU2TMF65e/YVSbSnSdc5BZu87Tv6l6sltRCpPvDn7H6bjTubbLaGNI3aJC8QoMrT00z7EcPHiQ5ORkAgMDc2zz4MEDBg4cyNq1a3F3d2fJkiV8/vnnzJw5k/HjxxMVFYW1tTXx8fG4uLjQs2dP3N3dGTRoUI7nNIYi37MA7V/8C+4V8fvpR9ze70n8ihVc7Pk+qbduZbbRCA09q/Rkfov5pGss6ObjzYJTS0i/c5WG5TxoVaUE03aoYreiKNm7evUqXbt2Zd68eWg0Of/qPXXqFBERETRr1oxq1aoxbtw4Ll3SDuWvVKkSXbp0YcmSJVjp/jFbUFTPQo/QaPD8+GOsAwO59vkXRLbvQMlfpmNTpkxmm2qe1VjVehVf7viEiexn//rOjHljNSNaVWTb6et8ve4kM7upacwVpbAwtAfwxDnintKdO3do1aoV33zzDS+++OIT20opqVq1Krt3Pz6jxKZNm9i5cye///4733zzDSdOnDB6rDlRPQvAf9FC/BctzPzs8vrrlFqwgPT794ls3yFL4RvA2caZyS/PZIRdWQ4+uM7bv79BdOJxVexWFOUxycnJvPHGG3Tr1o3Q0NBc2wcFBXH58mUOHjyYeXxERARpaWlER0fz0ksvMWHCBG7cuEFCQgKOjo7cvXvX1F9DJYuc2NeoTumVK7Dy8eFSr17ELVyUpfAthKBD88ksvXoDh5Qkev7Zk2THDZT2sFXFbkVRMq1cuZJdu3Yxf/78zOGx4eE5TqCNjY0Nq1evZvDgwQQHB1O9enUOHDhAamoqnTp1omrVqtSoUYOhQ4fi6OhI27ZtWblyJdWrVzdpgdukU5QXpKeZovxJ0u/f5/KnQ7m3dSsu7drh/fnIzMI3AGGDSAhfwtj6Xfj90hbKOlUh/EgrBjeprYrdimIm+Zmi3JS3oQqLp5miXPUscqFxcMDvxx9we/994leu5OJ7PbMUvvnfJ9gD39xLY2yDsVxJOIdLuR+ZdvB3VexWlGfIvBbznutE8bRUsjCAtvA9GJ/vxpH4zz9Etu/Ag3PntDud/SCkO4QvobVrZVa2XkmAc0msfBbwbtgIktOSzRq7oiiKMahkkQfObdtSaqFe4TtjtEKDwaCxhF0T8HfyZ0XrJVR1fI1rbKHtb+2JuhNl3sAVRVGekkoWeWRfXVf49vPjUu8PiFu4EOnoDTXfg6PL4OZZrC2smddmDE53enH53hXahbUj7FyYuUNXFEXJN5Us8sHK15eAJYsp9lITYsZ+y7UvvkTW6QuWtrDzOwCsLTV816Ijd88NxEkTwGd/fcaIv0aQkKLqGIqiPHtUssgnjYMDfj/8gFuvXsSvWsXFAcNJrdQNjq+C69ppAxqUc+fVihW4GNGVTuXfI+xcGO3XtedM3BkzR68oyqOiunYjqms3c4dRaKlk8RSERoPn4I/wGf8diUePEvnTER4kOMLOcZltRr5WEQuNJRf+a8Dsl2dzP+U+ndZ3Yvnp5Twvw5YVRXlcQUxRPm3aNJYsWWKS+B+lpvswAuc2bbAqWZLofv2J/NMV39sbKNbwBHhXpoSzHf1fKsd3G0/TsXYtVrdZzYi/RjDmwBgOXD3AV/W+wtnG2dxfQVEUE8mYorxcuXJcuXKFkJAQXnnlFdasWQPAjh07mDhxIuvWrcv2+NTUVCwts/9V3bdvX5PF/SjVszAS++rVKb1qJVal/Lm0qzhx336U2XPInMY8LAJ7C2emNZ3GJzU/YUf0DkLDQgm/nvPTnIqiPNvyM0W5n58fX3/9NfXr12fNmjX88ssv1KpVi+DgYEJDQ0lM1K7eOXLkSKZMmQJAgwYNGDZsGLVr1+aFF14w+tPcJu1ZCCFaAFMBC2C2lHJcNm3aAV8BEjgqpeyk2/4OMFLX7Bsp5QJTxmoMVj4+BCxbzuV33yBm0yUefNwX73FTsLa2ZnSbynSZcyBzGvN3Kr1DiFcIQ3YOofvG7vSr3o93K7+LRqj8rSjGdG3sWB6cyn2K8qTT2jaG1C1sKlbA+7PP8hyLIVOUZ3BwcGDPHu28dLGxsXzwwQcADBs2jPnz59OnT5/HjpFScvDgQdauXcvo0aPZuHFjnmPMicl+MwkhLIBpQEsgCOgohAh6pE05YDhQX0pZCRik214c+BKoA9QGvhRCuJoqVmPSODjgN28FblVTiN+wPfOJ7wbl3GlVpUSWNbsru2sf4mvu35ypf0+l9+be3Ey8aeZvoCiKKRg6RXmG9u3bZ74/duwYDRs2pEqVKixfvpyIiIhsj3nzzTcBCAkJITIy0ihxZzBlz6I2cFZKeR5ACLEcaAvoL1j9PjBNSnkLQEqZMV3rK8BmKWWc7tjNQAtgmQnjNRph74pnvw+wmTuRq0fCiWzXnpLTf2bkaxXZfuY6o9edZJZuGnNHa0fG/288L5Z4kXEHx/HW2rf4tsG31PPN29KKiqJkz9AeQEaPQn8GamPJyxTlGRwcHDLfd+vWjT/++IPKlSsze/bsHJdRtbGxAcDCwoLU1NSnD1yPKe95+AKX9D5H67bpKw+UF0LsEULs1922MvRYhBC9hBCHhRCHc7sHWODq9Ma5oh3+nX1JT0wkskNHHI8eYkDTcmw+GcP20w+nMRdC8Fb5t1jWahnFbYvTe0tvphyZQkp6ihm/gKIoxpDXKcqzc//+fby9vUlJSWHp0qVGjtAwpkwWIpttj44VtQTKAY2BjsBsIYSLgccipZwppawppazp4eHxlOEamY0jNBiEXdJeSk8ZhlXJklz6oA9vnttFoLs9X4U9Po15WdeyLG21lLfLv82cE3PovrE7l+9dNtMXyFmPjT0MWnpSUZS8T1GendGjR1O7dm2aN29OUFBQ7geYgMmmKBdC1AW+klK+ovs8HEBK+a1em1+A/VLK+brPW4FhQFmgsZSyt277DGCHlDLH21CmmqL8qSTfh6nB4FmR9NAVXBk6lLubt5D08muE2jZgwCtBDMhhGvONkRsZtXcUQghG1RtFc//mBRx8zorCVM7Ksy0/U5Sb8jZUYVFYpyg/BJQTQpQWQlgDHYC1j7T5P6AJgBDCHe1tqfPAJuBlIYSrrrD9sm7bs8XaQTvJ4IVdaGKO4Dt1Km4f9Mb2z3X8Ej6fhRvDc5zGvEVAC1a1XkWAUwCDdwzmm/3fkJSaVMBfQFGKjkdXzFSyMlmykFKmAv3Q/pI/BayUUkYIIUYLIdromm0CYoUQJ4HtwBApZayusP012oRzCBidUex+5tTsAY4lYPtYhBB4DhqEz4QJ+F47z4RtU5k2788cD/Vz9GNBiwX0qNSDFWdW0HlDZ87Hny/A4BVFUbRMOqhfSrlBSlleShkopRyj2/aFlHKt7r2UUg6WUgZJKatIKZfrHTtXSllW93p273dY2UHDj+HiXji/HQDn1q8RsGghrhbphM77ij2LH+1w6R1uYcXgmoOZ3mw6NxNv0mF9B9b8t0ZNFaIouVD/j2T1tH8e6gmwglCjGzj5wfaxoPsLswsOpvyvq4h19sBlzDBiZs994l9mA98GrGq9iqruVfli7xcM2z2M+yn3C+obKMozxdbWltjYWJUwdKSUxMbGYmtrm+9zqLmhCoKlDfzvE1g3CP7bDOVfBsC+pC/2v8xl74CPaTBxAmkXzlHiyy8R1tbZnsbT3pMZzWcw58QcpoVP48TNE4xvNJ5KbpUK8tsoSqHn5+dHdHR0rtNqFCW2trb4+fnl+3iTjYYqaIVyNJS+1GT4KQTsikOvHSAejg7uu+QwnqsW0u70Zuxr1sT3xx+wdH3yA+t/x/zNp7s+JTYplsEhg+lSsQtCZDfi2PjUaChFeX4UhtFQij5La2g0FK6Gw5kNWXaNfK0SK6q+yoa2fUg8dozI0HY8+O+/J56uhlcNfm3zKw19GzL+0Hj6b+vPraRbpvwGiqIUYSpZFKSqHaB4Gdj+LaSnZ24u4WzHgKbl+FEEcvObqaQ/SCKyQ0fu7tjxxNM52zgztclUhtUext4re3k77G0OXyvEvStFUZ5ZKlkUJAtLaDQMYo7DqawjoN6tX5pADwdGnJH4LF2OlX8povt8SOzceU8s0gkh6FyxM0teXYKdpR3v/fke049OJy09LcdjFEVR8koli4JW5W1wLw87vgW9X+jWlhpGt61MVGwCc/9NIGDxYhybN+f6+PFcHTESmZz8xNNWdKvIitdW0Kp0K34O/5mef/Yk5n6Mqb+NoihFhEoWBU1jAY2HwY3TELEmy676Zd1pVbUE07af5XIS+E6ZjPuHfbj9229EvfsuqXFPfi7RwcqBsQ3H8k39b4iIjSA0LJRd0btM+W0URSkiVLIwh6A3wDNI27tIyzqN8MhWFbHQCEavO4nQaPAYMACfSRNJOn6CyNB2JP37b66nb1u2LSteW4GnvSd9t/ZlwqEJpKSpGWwVRck/lSzMQaOBxsMh9iwcX5VlV0axe/PJGLad1t5Gcm7VCv9FC5HJyUQZUPgGKO1cmiWtltDhhQ4sPLmQrn905dKdS7kepyiKkh2VLMylwmvgXQV2fgeP/Ks/o9j91dqTmdOY21WtSsCqlVgHBGgL33Oe/MQ3gI2FDSNeHMGUxlO4ePcioetC+ePCHyb7SoqiPL9UsjAXjQaajIBbF+Bo1pnXM4rdF+MSmLnr4cSBVt7e+C9ehOPLL3N9wgSujhhJei6Fb4Cm/k1Z3Xo15VzK8emuT/ly75ckpGQ/262iKEp2VLIwp/ItwKcG7JygfcJbj36xW38ac429Pb6Tv8f9ww+5/dtvXOyRe+EbwKeYD/NazOP9Ku+z5r81dFzfkX9v5V7/UBRFAZUszEsIbe/i9kX4Z9FjuzOK3aPCTmY9TKPBY0B/fL+fRNIJwwvflhpLBtQYwIzmM7j94Dad1ndi5ZmVarI1RVFypZKFuZVtCn61YfckSMm6uFFGsXvLqYfFbn1Or76K/+JFDwvf27cbdMm6PnVZ3WY1IV4hfL3/az7Z+Ql3ku8Y5esoivJ8UsnC3ISAl0bAncvw94LHdmdX7NZnV6UKAatXYV26NNEf9iV2zhyDegrudu5Mbzadj0I+YtvFbbQLa8exG8eM8pUURXn+qGRRGJRuBP71db2LxCy79IvdM3Zmv0qelZeXtvD9yitcnzCRq5+NMKjwrREa3q38LvNbzgfgnT/eYe6JuaTL9CcfqChKkaOSRWEgBDT5DO7FwKE5j+3OKHb/vONsjmt2a+zs8P1+Eu59+3J7zRpt4Ts21qDLB3sEs7L1SpqUasLkI5P5cMuHxCYadqyiKEWDShaFRUADbQ/jr8mQ/PgKeDkVu/UJjQaP/v2yFr7PGDbiycnaiUmNJvH5i59zOOYwb4e9zf6r+/P9dRRFeb6oZFGYNBkBCTfh4MzHdpVwtmPgE4rd+jIL3ykpRHXsyN1thhW+hRC0e6EdS1stxcnaiV5/9uKHv38gNT0194MVRXmuqWRRmJSqA2WbwZ6pkPT46KQeuRS79WUpfPftS+zs2QYPkS3vWp5lrZbxRrk3mHV8Fu9ueper967m6yspivJ8UMmisGnyGSTeggMzHttlSLFbX2bhu8UrXJ84iavDPzOo8A1gb2XPqHqj+K7hd/x761/eDnubrRe35vnrKIryfFDJorDxDYHyLWHfj5AY/9ju+mXdeS2XYrc+beH7e9z79eP2//0fF7v3MLjwDfBqmVdZ+dpK/Bz9GLR9EGMPjFWjpRSlCDJpshBCtBBCnBFCnBVCDMtmf3chxA0hRLju1VNv33ghRIQQ4pQQ4gchhDBlrIVKk+GQdBv2/5zt7hEGFLv1CSHw6NcX3ymTSTp5Ulf4PmNwOKWcSrG45WK6BnVl2ellnIo7RVJqUu4HKory3DBZshBCWADTgJZAENBRCBGUTdMVUspqutds3bH1gPpAVaAyUAtoZKpYC50SwVCxNez7GRIen/dJv9i99ZThq+E5tWiB/6JFyNRUojp24u62bQYfa2Vhxae1PuWnl34iJS2Fk3EnCTsXZvDxiqI820zZs6gNnJVSnpdSJgPLgbYGHisBW8AasAGsgKK1RmjjzyD5Huz9MdvdPeqXpqxnMUaF5V7s1mdXpbJ2qvMyZYju2y9PhW+ARiUbEeQWhL2lPZ/99Rkj/xqpZrBVlCLAlMnCF9BfbSdat+1RbwkhjgkhVgshSgJIKfcB24GrutcmKeWpRw8UQvQSQhwWQhy+ceOG8b+BOXkFQaU3tIXu+zcf221tqWF0m0oGF7v1WXl54b9o4cPC97DhBhe+AawtrHnB9QU+CP6AtefW0mF9B87EGX5bS1GUZ0+ek4UQQiOEcDKkaTbbHv0nbBgQIKWsCmwBFuiuURaoCPihTTAvCSH+99jJpJwppawppazp4eGRl6/xbGg8HFITYc+UbHfXy2OxW19m4bt/P27//jsX3+mep8K3EIK+1foy6+VZ3E2+S+cNndUMtoryHDMoWQghlgohnIQQDsBJ4IwQYkguh0UDJfU++wFX9BtIKWOllA90H2cBIbr3bwD7pZT3pJT3gD+AFw2J9bniUR6qhMLB2XA3+7twI1sF5anYrU8IgUdfXeH71CkuhIbmqfANUKdEHVa3zjqD7d3ku3mORVGUws3QnkWQlPIO8DqwASgFdM3lmENAOSFEaSGENdABWKvfQAhRQu9jGyDjVtNFoJEQwlIIYYW2uP3YbagiodFQSEvWTgOSDW9n23wVu/U5tWiB/+LFkJpGZMdO3N365OcpOvwYQYcfIzI/u9m5Mb3ZdAbVGMTWi1sJDQvlxM0T+YpFUZTCydBkYaX7pf068LuUMoXHbyllIaVMBfoBm9D+ol8ppYwQQowWQrTRNRugGx57FBgAdNdtXw2cA44DR4GjUsqiOfTGLRCCO8LhuXDnSrZN8lvs1mdXuRIBq1ZhExhIdL/+3Jw5K0+3lDRCw3tV3mN+i/mky3S6/tGVBREL1G0pRXlOGJosZgCRgAOwSwjhD+S6Wo6UcoOUsryUMlBKOUa37Qsp5Vrd++FSykpSymApZRMp5Wnd9jQpZW8pZUUpZZCUcnB+vtxzo9EQkGnaKcyz8TTFbn1WXp74L1qIU8sW3Pj+e64OG0b6gwe5H6inmmc1VrVexf98/8fEwxPpt60ft5Ju5TsmRVEKB4OShZTyBymlr5TyVakVBTQxcWxKBtcAqN4FjiyA+EvZNtEvdl+Mzf9QVo2tLT4i68/eAAAgAElEQVSTJuE+oD+3f1+rLXzffHw01pM42zgzpckUhtUexr4r+3g77G2OxBzJd0yKopifoQXugboCtxBCzBFC/A28ZOLYFH0NP9Gue7FrQo5NMordo9dF5NjGEEIIPD78EN8pU0g6fZoL7dqRdPp0ns/RuWJnFr+6GFsLW97d9C4zjs4gLT1/t8kURTEvQ29DvasrcL8MeAA9gHEmi0p5nEtJqPEOhC+BuAvZNnlY7L6e72K3PqcWr2gL32npRHbqzN0tW/J8jiC3IFa2XkmLgBb8FP4TvTf35kbCc/ZMjKIUAYYmi4xnJl4F5kkpj5L9cxSKKTX8GITFE3sXGcXur8Ii8l3s1qctfK/UFr77D+DmjJmQx6K1g5UD4xqOY3S90Ry9cZS3w95m7+W9Tx2boigFx9BkcUQI8SfaZLFJCOEIqKlHC5pTCaj1HhxdBrHnsm2SUey+FJfILzuzb5NXVp4Zhe+W3Jg8GbfrSZCet4QhhOCNcm+wrNUyitsWp/eW3kw5MoWU9BSjxKgoimkZmizeA4YBtaSUCWjnbOphsqiUnDX4CCxsYEfOdwEzit3Td5x7qmK3Pm3heyIeAwfgcC8Vz6uJpN2+nefzlHUty9JWS3mr3FvMOTGHdzeqhZUU5Vlg6GiodLRPYI8UQkwE6kkpj5k0MiV7xTyh9vtwfBXcyPlpa2MVu/UJIXDv04ebXrbYJKUR2bkzKVfz/oveztKOr+p9xfj/jee/+P94O+xttl00fAZcRVEKnqGjocYBA9FO9XES7cN035oyMOUJ6g8CawfYkfNfgbezLYOaGa/YrS+hmBXXfexIvRZDZIeOJP37b77O07J0S1a+thLfYr4M3D6QcQfHkZxm+ISGiqIUHENvQ70KNJdSzpVSzgVaAK1MF5byRA5uUKc3RKyBmJx7DsYudut7YGeJ/+JFkJ5OVOcu3D94MF/nKeVUisWvLqZLxS4sObWELhu6cPHORaPGqijK08vLrLMueu+djR2Ikkd1+4GNE2wfm2MTKwvjF7sBEpLTSEhOw7ZCBQKWL8PSw4NL7/XkzsZN+TqftYU1Q2sPZWqTqVy+d5l269qx4fwGo8WrKMrTMzRZfAv8I4SYL4RYABwBcv4tpZiefXF48UM4vQ6uhOfYrF5Zd1oH+/CzEYvd+qx8ffFfshjbypW5/NFHxC1anO9zvVTqJVa3Xk05l3IM3T2Ur/Z+RWJqohGjVRQlvwwtcC9DO0X4b7pXXSnlclMGphig7odg6/zE2gXAiFcrYmXkYrc+S1dXSs2bS7GXXiJmzBiuT5qU7wkESxQrwdwWc+lZpSe//fcbndZ34uyts0aOuGD12NiDHhvV4EHl2fbEZCGEqJHxAkqgXaPiEuCj26aYk60z1OsP/26E6MM5NvN2tmWgrti95aRpVqfV2Nri98NUXDq0J3bWbK4OG4ZMyd8zFFYaKwbWGMgvzX4hLimOjus78tt/v6kZbBXFjHLrWUx6wmuiaUNTDFLnA7Ar/sTaBWiL3eU8izFqnfGL3RmEhQXeX36Jx8AB3P59LZc+6EPavfv5Pl8933r82uZXgj2D+XLvlwzdPZR7yfeMGLGiKIZ6YrLQTRue00tNJFgY2DhC/YFwbitc3J9jMysLDaPaGqfYPa5TAOM6BWS7L+NZjBJjvuH+/v1c7NYtz7PW6nO3c2dGsxn0r96fTZGbaL+uPSdj874qoKIoT8fQ5yzezObVVAjhaeoAFQPUfh8cPGD7mCc2qxdo2mK3Ppe33sJv2k88uHCByI6dSI6MzPe5LDQW9Krai7mvzCUpLYkuG7TDbNVtKUUpOHmZ7mM20Fn3mgUMBvYIIXJbXlUxNWsH7TQgF3bBhd1PbJpR7B4VZppitz7Hxo3xXzCf9Hv3iOzYicRjT/fQf4hXCKtbr6aeTz3GHRzHoO2DuP0g71OOKIVb+xn7aD9jn7nDeGbUmfcWdea9ZfLrGJos0oGKUsq3pJRvAUHAA6AOMNRUwSl5UPNdKOatrV084V/cGcXuradNV+zWZ1e1Kv5Ll6BxcCDqne7c27nzqc7nauvKjy/9yJCaQ9h1eRehYaGEX8956LCiKMZhaLIIkFLq/2a5DpSXUsYBatrQwsDKTjuF+cW9cH77E5sWRLFbn03p0gQsX4ZN6dJc+rAv8b/+9lTnE0LQrVI3FrVchEZo6L6xO7OPzyZdFs6JkE9evcPJq7muQqwohZqhyWK3EGKdEOIdIcQ7wFq0a3E7APGmC0/Jk5B3wMk3196FMYvdhrJ0d6fUwoU41KnD1REjuDl9+lPXHCq7V2ZV61U082/G1L+n0mdLH24m5r+YrhQOkdYTibRWgy0LG0OTRV9gHlANqA4sAPpKKe9LKdVa3IWFpQ387xOIPgT/bX5i04IsdmewKOZAyV+m49SmNTem/sC1UaOQaU/Xs3G0dmTC/ybwRd0vOBJzhNCwUA5cPWCkiBVFyWDoE9wS+AvYBmwBdkk1FKVwqtYFXEppR0bl8leU32J3QPInBCR/kq/whLU1Pt99h9v7PYlfvoLogQNJT0rK17kyzykEoeVDWdpqKY7Wjrz/5/v89M9PpKanPtV5FUV5yNChs+2Ag8DbQDvggBDibVMGpuSTpTX871O4Gg5nnjwZX0EXuzMIIfD8+GO8Rozg3tZtXOzxLmnxT383s7xreZa3Wk7bsm2ZcWwG7216j2v3rxkhYkVRDL0NNQLtKnnvSCm7AbWBz3M7SAjRQghxRghxVggxLJv93YUQN4QQ4bpXT719pYQQfwohTgkhTgohAgyMVQnuCK6lYfu3kP7kom9BF7v1Fe/aBd/J35N04gSRnTqTcvnyU5/T3sqer+t/zdgGYzkVd4rQsFB2Xnq6EViKohieLDRSyut6n2NzO1YIYQFMA1qiHWrbUQgRlE3TFVLKarrXbL3tC4EJUsqKaJPT9WyOVbJjYQmNh0HMcTi19olN9Yvd03cUTLFbn1OLFpScM5vUGzeI7NiJpDM5r/6XF60DW7PytZV4O3jTb1s/JhyaQEqaGrinKPllaLLYKITYpOsJdAfWA7ktOFAbOCulPC+lTAaWA20NuZguqVhKKTcDSCnv6db+VgxVJRTcymlnpE1/co8ho9g9fWfBFbv1OdSujf/ixSCEdiGl/cYpUAc4B7D41cV0eKEDC08upNsf3bh095JRzq0oRY2hBe4hwEygKhAMzJRS5vYwni/aGWozROu2PeotIcQxIcRqIURJ3bbyQLwQ4jchxD9CiAm6nkoWQoheQojDQojDN27cMOSrFB0aC23v4sZp7Yp6uSjIJ7uzY/tCee1CSt5eXHr/fe5sMM7iRzYWNox4cQSTG08m6k4U7cLasTFyo1HOrShFicEr5Ukpf5VSDpZSfiSlzP23D4jsTvPI5zC0D/xVRTvKaoFuuyXQEPgEqAWUAbpnE9NMKWVNKWVNDw8PA79JEVLpTfCoqO1dpD15ZJB2ze7yBV7s1mdVogQBS5ZgG1yVy4M/Jm7BgtwPMlAz/2asarOKMs5lGLJzCKP3jSYp9elGYSlKUZJb3eGuEOJONq+7QojcHkmNBkrqffYDrug3kFLGSikf6D7OAkL0jv1HdwsrFfg/QK2fkVcaDTQZDrFn4fiqXJt3rx9gtmJ3BgtnZ0rNmYNj8+bEfDuOmPETkLkU6Q3lW8yX+S3n06NyD1b9u4pOGzpx/vZ5o5xbUZ53uU1R7iildMrm5SildMrl3IeAckKI0kIIa6AD2ie/MwkhSuh9bAOc0jvWVQiR0V14CVDzUudHhdbgXQV2fge5FHjNXezOoLGxwXfKZFw7dSJu7lyufDoUmZxslHNbaawYHDKYn5v+zM2Em3RY14Hfz/5ulHMryvPM4NtQeaXrEfQDNqFNAiullBFCiNFCiDa6ZgOEEBFCiKPAAHS3mqSUaWhvQW0VQhxHe0trlqlifa5pNND4M7h1AY4uy7V5vUB32uiK3VGx+V+46GkJCwu8Ph+Jx0cfcWfdOi598AFp94y38FFDv4asbrOayu6VGblnJJ/t/oyEFDWGQlFyYrJkASCl3CClLC+lDJRSjtFt+0JKuVb3friUspKUMli3oNJpvWM3SymrSimrSCm760ZUKfnxQkvwqQ47J0Bq7n+MI1rp1uwOM29nTgiBe+9elPj2W+4fOEhU126kXDfeCGpPe09mNZ/Fh8Efsv7Cetqva8/puNO5H6goRZBJk4VSSAgBTUbA7Yvwz6Jcm3s5mb/Yrc/ljdcp+ct0kqOiiOrYiQfnLxjt3BYaC/pU68Psl2dzP+U+ndd3Zvnp5WphJUV5hEoWRUXZZuBXG3ZPgpTcRwFlFLu/CjNfsVtfsYYN8V+wgPTERKI6dSIx3LhrWNTyrsXqNqupXaI2Yw6M4eOdH3MnWU0rrigZVLIoKoSAl0bAncvwd+5DUjOK3dG3zFvs1mdXpTIBy5aicXIiqnsP7m578rodeVXctjjTmk5jcMhgtl/cTruwdhy78XSr+ynK80Ili6KkdCPwr6/rXSTm2rywFLv1Wfv7E7BsKTZlyxLdrx+3VuU+JDgvNEJDj8o9mN9yPlJK3vnjHeafmF9oF1ZSlIKikkVRIgQ0+QzuxcChOQYdklHsHmXmYrc+Szc3/BfMx6F+fa59/gU3fppm9BpDsEcwK1uvpHHJxkw6Mom+W/sSlxRn1GsoyrNEJYuiJqCBtofx12RIzr23kFHs3lZIit0ZNA4OlPx5Gs6vv87Nn37i2hdfIlONu36Fs40z3zf+nhF1RnDg6gFC14Zy6Noho15DUZ4VKlkURU1GQMJNODjToOaFrdidQVhZUeLbsbj17k38qlVE9x9AemLut9fydA0h6FChA0tbLcXeyp6ef/Zkevh00nKZnFFRnjcqWRRFpepoR0ftmQpJuY/4sbLQMLptZaJvJfJzISl2ZxBC4PnRILy++Jx7O3ZwsXsPUm/dMvp1KhSvwPLXlvNq6Vf5+ejP9Nrci+sJatZ8pehQyaKoavIZJN6CAzMMal430I02wT78svNcoepdZCjeqRO+U6eQdOoUUZ06kxz99AspPcrByoGxDcbydf2vOX7zOKFhofx1+S+jX0dRCiOVLIoq3xAo3xL2/QiJhi1pmlHsjjLDmheGcHr5ZUrNm0tqbCyRHTuQdOpU7gflkRCC18u+zvJWyyluW5w+W/rw/ZHvSUlXCyspzzeVLIqyJsMh6Tbs/9mg5hnF7vjEFC7cvM+G41eJuVO4pvm2DwkhYOkShKUVUV26cn/fPpNcp4xLGZa1WkZo+VDmnZhH943duXzP+L0ZRSksVLIoykoEQ8XWsO9nSDBsWGj3+gEUt7fixr0HfLjkb+qM3Ur9cdvov+wf5u+5wLHoeFLSzPtMgk3ZsgQsW4qVjw8Xe/Xm9rr1JrmOraUtX9T9ggmNJnA+/jyhYaFsjdpqkmspirlZmjsAxcwafwan1sHeH6HZl7k2t7LQUM7LkXQpGdEqiL+jbnHk4i0OR8YRdlS7XImtlYaqfi6E+LsSUsqVGv6uFHewNvU3yRqntzf+SxYT/WFfrnzyCanXr+P2bg+TXKtFQAsquVViyM4hDNoxiI4VOvJxzY+xsbAxyfUUxRxUsijqvIKg0hvaQnfdvuDgbtBhGiGoVtKFaiVdeJfSAFyJT+Tvi7c4EnWLvy/GM2vXeaanax+WK+3uQI1SrtTw1yaRcp6OWGiyW0zReCycnCg5ZzZXhg7j+vjxpMbE4Dn0U4TG+B3qko4lWdRyEZP/nsyik4v45/o/TPjfBAKcA4x+LUUxB5UsFGg8HE7+H+yZAi9/k+/T+LjY4eNix2tVfQBISknj+OXbHInSJpCd/17n17+jAXC0saRaKRdqlHIlxN+VaqVccLK1MsrX0aexscH3+0nEeHgQt2ABqTduUGLct2isjd/TsbKw4tNan1LbuzYj94yk/br2fF73c6NfR1HMQSULBTzKQ5VQODgb6vYHRy+jnNbWyoJaAcWpFVAcACklF+MSdD2PWxyJiufHbf+RLrUzkZT3dKSG/8MEUtrdASGevvchNBq8PhuOlZcn1ydOIjU2Fr+ffsTC0fGpz52dxiUbs7r1aobuGsrw3cOxEE5YS8N6bIpSWKlkoWg1GgrHV2unAWk5ziSXEELg7+aAv5sDb9bwA+Deg1SOXorP7H2sP3aVZQcvAeBqb6W7deVKjVKuBJd0xt46f//JCiFw69kTSw8ProwYSVSXrpScORMrL0+jfT993g7ezHllDtOPTmfm0Zkkau7QaEUjAl0CKeNchrIuZQl0CSTQJZDitsVNEoOiGJNKFoqWWyAEd4TDc6H+AHDyKZDLFrOxpH5Zd+qX1f7LOz1dcu7Gvczax5GoW2w9rX1S2kIjqFjCMbNoXqOUK36udnnqfTi3bYuFmzuXBwwgsmMHSs2ahU1goEm+m6XGkv7V+7MwfCNpMpHGJRtyLv4c68+v517KwyViXW1cMxNHRiIp41IGN1s3o/SsFMUYVLJQHmo0BI4t105h3mqSWULQaATlvBwp5+VI+1qlAIhPSOafi/GZt69WHYlmwb4oADwdbQjRJY4a/q5U9nXCxtLiidco1qA+pRYt5FKv3kR16ozf9OnY16huuu+ELRpsGVVvFKC9HXc94Trn4s9x7vY57c/4c2w4v4G7KXczj3OxccmSPAJdAinrUlYlEcUsVLJQHnINgOpd4MgCqD8IXEqaOyIAXOytaVLBkyYVtLeMUtPSOX3tLv/ojbz648Q1AKwtNFT2dcpMICH+rng62T52TrtKlQhYvoxLPd/nYo8e+H4/CcemTQvk+wgh8HLwwsvBi3q+9TK3Sym5kXgjM3lkJJI/Iv/gbvLDJOJk7ZSZQMq6lKWMszaReNh5qCSimIxKFkpWDT+B8KWwawK0+cHc0WTL0kJDZV9nKvs607VuAADX7ybxd1R8ZgJZsC+KWbu1a3X7udplJo4apVypUMIRKwsN1iVL4r9sKZc+6EN0/wF4f/EFrh3am+17CSHwtPfE096Tuj51M7dLKYlNiuVs/NmHiST+HJujNrP639WZ7RytHQl0Dsy8pRXoEkigcyCe9p4qiShPTSULJSuXklDjHTgyDxp8BMVLmzsig3g62tKisjctKnsD8CA1jZNX7mTeujpwIZa1uocG7awsCC7p/HDY7s8zsBgxlGtffUXq9Rjc+/cvVL9chRC427njbufOiyVezNyekUTOx5/nbPxZzt/W/tx6cSu//vdrZjtHK8fM21j6ycTL3qtQfU+lcDNpshBCtACmAhbAbCnluEf2dwcmABmT6vwkpZytt98JOAWskVL2M2Wsip6GH8PfC7W9i9cNmzeqsLGxtKB6KVeql3IFtL9Yr9xO0j5xrksgM3edJ1X30GDZwHb0u6uBn6dzMzKa8uPGYGFt/Oc+jEk/idQuUTvLvrikuMweSEYi2XFpB7/991tmGwcrh2x7It4O3iqJKI8xWbIQQlgA04DmQDRwSAixVkr56PqcK56QCL4GdpoqRiUHTiWg1ntw4Bdt4nAzzWihgiSEwNfFDl8XO1oHa0d6JSancSw6niMXb/F3VDyjK7xBq/uWdNoQxrLwc2wLHUjVct7a3kdJFxxN8NCgqRS3LU5x7+LU8q6VZXtcUhzn489nqYnsit7FmrNrMtvYW9o/NjKrrEtZvB280Qg1nVxRZcqeRW3grJTyPIAQYjnQFjBoMWchRAjgBWwEapoqSCUHDT6Cw/Ngxzh4a5a5ozEJO2sL6pRxo04ZN0Db+4iMrc/5uYuoPu8H3Jd9y9Ca3ZliXQwh4AUvx8whuyH+rgS42T9z/wLPSCI1vbP+LxWfFJ9lZNa52+fYc2UPv5/7PbONnaUdgc6BWUZmlXEug08xH5VEigBTJgtf4JLe52igTjbt3hJC/A/4F/hISnlJCKEBJgFdgYIZoqJkVcwTar+vnWDwf5+AxwvmjsjkhBCUdneg9KcfcDekHBaDP2bViXnc+Pw7Dj2w4++L8YSFX2HpgYsAFHewpkYpl4cPDfq5YGf95GG7hZWLrQshtiGEeIVk2X77we3M5JFRG9l/ZT9rz63NbGNnaUdp59JZRmYFugTiW8xXJZHniCmTRXb/5JKPfA4DlkkpHwghPgAWAC8BHwIbdIkj5wsI0QvoBVCqVCmjBK3oqT9I+5Dejm8hdH7m5i9ih+jePb+rxDk2bUqpefOI7tMHj+H96D3jF+ya1SY9XfLf9YcPDf4ddYstp7QPDVpqBBVL6Ibt+rtSo5QLvi52Zv4mT8fZxpkaXjWo4VUjy/bbD25z4fYF7QitW9pk8mgSsbGwxd8pgADHMgQ4laGUY2lKFSuNp70PAg3pUiIlpEtJuu6nlJLUBy4AnLl2F6n3K0PKrD+BbPfry65t1m36bWU227KcLcfjs2zT+5Dtfh4/QU7XzC3m9PR0ku6XQGiSMTUhs/sTNsaJhagLfCWlfEX3eTiAlPLbHNpbAHFSSmchxBKgIZAOFAOsgZ+llMNyul7NmjXl4cOHjfwtFLaO1j6k12cveFUCIGJsAwAqffb8JosMD86d4+L775MefxvfH36gWIP6j7WJu5/MPxdvZSaQo5duk6hbetbLyYb49NNoLBN5q0IL0qX2l0l6+sNfklLKR35hZuzT359N+/Q8ts/u/Ol5bP/I+R/79aFJQmMTg4X1dTQ2MWhsdD+tbmc2kemWpCd7kP7AS/fyJO2BFzKlOKZfYkcC6SDSQKQjRJr2vW6b0G2HR/brtmXu120XPHyf2Z6MNo9+fvT8aQiRnvP5STfgetq1Y9KTShDR+898/YkIIY5IKXO91W/KnsUhoJwQojTa0U4dgE76DYQQJaSUV3Uf26Ad+YSUsrNem+5AzSclCsWE6vaDg7Ng+1josMTc0RQ4m8BAApYt51KvXlz64AN8xo7BuU2bLG2KO1jTtKIXTStqJ2DMeGgwc76rk+6kJ1ix4fg1NEJ7u0sjtNO8a4RAZL7nkc967zUZ+/SP1Z1LA5ZCk+25NJrcrqXf1pDYsrl+xjZNzu1T0hO5lXKJ2JSLxCZfIvbBJW4mR3E7JTzzz9FSWOFhU5KrdxJAWlHHtyppMpV0mab7mUqaTMv8mSZTs+x/uC+VdNKyfE7LPEfBrR+vQYOFxhILkfGyePhe8+hnayx1bTTCAstHj9NY6vZbPnbOsP+2YaEx/VMQJruClDJVCNEP2IR26OxcKWWEEGI0cFhKuRYYIIRoA6QCcUB3U8Wj5JN9cXjxQ9g5Dq6Eg081c0dU4Ky8PPFfvIjofv258ulQUq9fp/h77+VY3NZ/aPCdegEcmKed9v1Aj1+zbV90BD225X7Kfe3oLL2ayLXEfUjSOXsvAUuN5cOXePjeRmOJpcYKS40dlhpLrIRV1ra6l4WwwEpjleN5LDWWWfc/su+xz/rH5XDNjOMKavDD5gt7C+Q6Jk1HUsoNwIZHtn2h9344MDyXc8wH5psgPMVQdT+EA9O1tYtOK8wdjVlYODpSctZMrg4bxvWJk0iJuY7XsKEIi2ezoF1YOFg5UMWjClU8qmRuqzPvLQB2tC/qybVwUU9wK7mzdYZ6/WHbNxB9xNzRmI3G2hqfiROx9PDULqR0/To+479DY6OWT1WefypZKIap8wHs+xm2jzF3JGYlNBq8hg/D0suL6+PHcykuDr9pP2Hh5JTjMcOWRmrfmGYJcEUpEGoQtGIYG0eoPxDObcUu/b65ozE7t3d74DNhAgnh4UR17kLKtWvmDklRTEolC8Vwtd8HBw8802LMHUmh4Nz6NUrNnEHKlStEdujIg//+M3dIimIyKlkohrN2gAYfUUzexyUtDtJSzB2R2TnUrYv/4kXItFQiO3ch4UjRrekozzeVLJS8qfkuScIG37TL8H0QbPkK4s6bOyqzsq1YkYBly7F0c+Nij3e582f+Ho5SlMJMJQslb6zsOGdZjihLf/CrCXumwg/VYWFbiFgDqaafdqAwsvbzxX/pEmwrVuTywEHELSl6DzAqzzeVLJS8E4J7GifouAwGnYAmIyD2HKzqDt9XhD8/134uYixdXSk1fx7FGjcm5utvuD55CqaaTkdRCpoaOqs8HWdfaPSpdt2Lc9u1K+ztmwZ7f4CAhhDSHSq2Bsui8SyCxs4Ovx9/4Nqo0cTOmEFqTIx2FrhnbCpzRXmUShaKcWgsoFwz7evuNfhnsXa1vV/fA7viUK2TdrlWj/LmjtTkhKUl3qNHYentxc0ff8LPVsMVt6KRLJXnl7oNpRifo7d2DYwB4dB1DZRuqF11b1otmPcqHF0BKYnmjtKkhBB49O2L99ejcUhKp+yVRKL7D+B2WBhpd+6YOzxFyTPVs1BMR6OBwJe0r3vXIXwpHJkPa3rBH59CcEcIeQc8K5o7UpNxDQ1l/49jcE5Iwzo8nLubN4OVFQ4vvohj82Y4Nm2KpZubucNUlFypnoVSMIp5QoNB0P9v6LZWm0AOzYafX4Q5L2sTSXKCuaM0iSQbC2JcrSm7cwf+y5ZSvGtXkiMjufbFl/zXoCGRXboQt2ABKZcvmztURcmR6lkoBUujgTKNtK/7N+HoMm1v4//6wB/DILi9trbhXdnckRqd0Giwr14d++rV8RzyCQ/OnOHu5i3c3byZmG/HEfPtOGwrVcKxeXMcmzfDJjDQ3CErSiaVLJQ8q1TC2TgncnDXzmZbtx9E7dEmjSML4OBM8K2pHUlV+U3tk+PPGSEEthUqYFuhAh79+5EcGcndLVu4s3kzN6ZM4caUKViXKaNLHM2xrRRUYOsjKEp2VLJQ8q7HeuOeTwgIaKB9tYyDo8u1iWNtP9g4HKq209Y2SgQb97qFiHVAAG49e+LWsycpMTHc3bKFu5u3EDt7NrEzZmDpUwKn5s1xbNYMuxo11DoaSoFTyUIpXOyLaxdberEPXNwPfy+A8CVweA74VNf1Nt7SzoL7nLLy8qJ4584U79yZ1Fu3uLdtO/PivCIAABB4SURBVHe3bOHWsuXELViIRfHiODZtiuPLzXGoUwdhbW3ukJUiQCULpXASAvzral8tvoVjK7W9jbCBsGmENmGEdNcmkOf49oylqysub72Jy1tvknbvPvd37+Lu5s3cWb+e+FWr0BQrRrHGjXFs3pxiDRugsbc3d8jKc0olC6Xws3OFOr2hdi+IPqxNGsdWansd3lW1SaNKKNjmvADR88CimANOLVvi1LIl6Q8ecH/fPu5u3sy9rdu4s24dwsYGh4YNcGzWDMcmTbBwNlJtSVFQyUJ5lggBJWtpXy3GwvFVcHg+rB8Mf47UFsNDesD/t3fnUVaUZx7Hv7++vbA0m9AgsipCFFTQacBBQcQGHSPgNsCo4xI9juNxZsyYzMGYoxLNZDSZyZy4TGKiEzViPKggCqI0CsbMgCKyI+toJCibgqAgNP3MH281Nr1Vd9N1b9P9fM6pc6vufavquW/fe59+36p6q9tfNOnWBkBWXh5tRo6kzciR2JQSvlr8HnvmzmVPcTF7i+fxSXY2rYcMoc2Y0eSPGkVO586ZDtkd4zxZuGNTi3Yw+CYovBG2LAmtjRUvhGFGupwWTr89YwK0bJ/pSPm3q3oDcFlC21d2Nq3PHkrrs4fS5a4fsH/lytBV9frrfHrvFJjyI1oOGnT4lNzcHj0SisQ1ZX5Rnju2SaElMe4h+N5auOQ/ISsbXv0+/PspMP3v4U+LwmB+zYCysmh5xhl0vuMO+syZw4kzX6LTP9xG6f79bHvwQTaOHsOmyy5n+6OP8vX69T4qrqs1b1m4piOvDRTeEKYtS8MxjeXTYNlUKDg1nH57xsRwxlUzIIkW/frRol8/Cm69lQMffxwuAiwuZsdDD7PjFw+R26sXbcZE13Kcfrpfy+GqlWjLQtJFktZK2iBpchWvXy9pu6Sl0XRT9PwgSf8raZWk5ZImJhmna4JOGASX/Bzu+CC0OnJbwZzJobXx4s3w0f80m9ZGmdwePej4nRvoPfUZTl4wn+PvvYecbt3Y+d+/5cMJE9lw/ig+vf/HfLlwEVZSkulwXSOTWMtCUgp4BBgNbAbelTTTzFZXKPqcmd1W4bmvgGvNbL2kE4D3JL1mZruSitc1UXn5cNa1Yfp0RbhCfPlzYerULxzbGPg30Dq5wfx6H/heYtuur5zOnekwaRIdJk3i0K5d7Jk/nz3FxeyaNo3Pf/c7Uu3bk3/BKNqMHk3rYcPI8ms5mr0ku6GGABvMbBOApN8D44GKyaISM1tXbn6LpG1AAeDJwtXf8afDt38Go6fAqhnhoPjrd8G8KXDquNBN1Xt4g59JdffO70dzbzfodhtKqn172l96Ke0vvZTSr75i7x/eDmdWvfY6u194kazWrck/7zzajC6i9fARpPKTHX5l8tQPw8wNie7G1VGSyaIb8HG55c3A0CrKXSFpBLAO+K6ZlV8HSUOAXKD53afTJSO3NZx5dZi2rg7HNpY9Cyufh+P6hKQx8CrIL8h0pGmX1aoVbS8cQ9sLx1B64ABfLVrEntfnsmfePL6YPRvl5tL6nHPCRYDnjyS7Q4cGjyHPmva9ThraPc+sCTMJJ9ckk0VV/55V7CR+GXjWzL6WdAvwJDDq8AakrsDTwHVmVlppB9LNwM0APXv2bKi4XXPSpT/81QNQdC+sfil0U829G+bdB6d8O1zwd+J5YbTcZiYrN5f84cPJHz6c4++9h31LlvBF2bUcb74JqRStBg8O9+UoKiKnS5dMh+wSlGSy2AyUP6G7O7ClfAEz21lu8dfAA2ULktoCs4AfmtnCqnZgZo8BjwEUFhY2r6OVrmHltISBk8K0fW1IGsumwuoZ0KF3OLYx6Gpo0zx/EBUlhlaDB9PlzjvZv2p16KqaO5et993P1vvup+XAgeHMqqIicnv1ynTIroEl+e/Su0BfSSdKygUmATPLF4haDmXGAWui53OB6cBTZjYtwRidq6zgW+EK8X/+AK54HNr1CMc1ft4fnrsGNhRDaaWGbrMhiZanDaDzd2+nz+xZnDTrFQpuvx0rKWHbT3/GxgsvYtO48Wx/6GH2r13r13I0EYm1LMysRNJtwGtACnjCzFZJ+hGw2MxmAv8oaRxQAnwGXB+tPgEYAXSUVPbc9Wa2NKl4naskpwWcfmWYdqyPRsCdCmtehvY9wxlWg66Btl3jt9WE5fXpQ16fPnS65e84+Oc/H74vx45HH2XHI4+Q07Pn4a6qlgMHombYpdcUqKlk/cLCQlu8eHGmw3BNXcnX8MGscCbV/y0ApaDfReHYxskXQFbl+0ys+tdzARjwg8Z5NlRSSnbsYM8bb7BnbjFfLlwIBw+SXVAQEsfo0bQqLEQ5OZXWm13UH4CLi2NPnHQcfX1Jes/MCuPK+RXcztVFdl4YsPC0y2HnRljyVLjfxtpZ0LY7nPW3cOY10K57piPNuOxOnegwYQIdJkzg0BdfsHfBAvbMLWbX9Bl8PvVZUu3akT9qVDgld9gwslq0yHTIrgaeLJyrr459wjUb598F614NrY35P4EFD0DfMVFrY3Smo2wUUm3b0m7sWNqNHUvpvn18+cc/hgPk8+axe/p01KoV+SNG0GZ0ESo1LMuHHWlsPFk4d7Syc6H/+DB9/iEseRrefxrWzYE2XSkoOcT+rJbwwWxQVrjoT1mAovnyyxVfr6Z8XcrGbfuI12q77fLl6yarZctwz42iIuzgQb58551oePV57Jkzh17AwWxj09ixoVsvlYVS2eFYRyoVHrPjllOobN2sVKVlZaeO3HYqC7JS4bHScircxrbSclZ4rLB8+LmscmUPL2d/s41aLacazXhdfszCuSQcOgjrXoP3fottmFvlRUdNR3WJqqrEUn3iMRP7tonPV++lZF+KVF5LrBQwvnk0oNTC4+Flql6OHsvmy14/clvpr606K6u2CtVbVnVmh8jKP8TJf6jfdct+zMK5TErlwKmXwKmXsO7HQ8m2Evp85/Ejf6mO+OUqjVmmDmXruu1alj88n0wswmh1svHZnhdJAd3HXJiWP5WZRYnFQiIqDTHbIYtCjJZLDQ7ZEcthnsPrHVG24jar2kcplZaP3MY3+6gurr2bP6IkDb/kniycS1iJcihRThgJ18Va/vB8ALr/1y/Tsj9VeDzWzC7qDwanJLwfP+HZOedcLE8WzjnnYnk3lHMJG9C1XaZDOKYMUF6mQzimpKu+vGXhnHMulicL55xzsTxZOOeci+XJwjnnXCw/wO1c0m6YlekIjim9rjoh0yG4KniycM41Lp5c66TX3PfTsh/vhnLOORfLk4VzzrlYniycc87F8mThnHMulicL55xzsTxZOOeci+XJwjnnXCxPFs4552J5snDOORdLZpbpGBqEpO3AR0exiU7AjgYKpyF5XHXjcdWNx1U3TTGuXmZWEFeoySSLoyVpsZkVZjqOijyuuvG46sbjqpvmHJd3QznnnIvlycI551wsTxbfeCzTAVTD46obj6tuPK66abZx+TEL55xzsbxl4ZxzLlazTRaS/lrSKkmlkqo9i0DSRZLWStogaXIa4jpO0lxJ66PHDtWUOyRpaTTNTDCeGt+/pDxJz0WvL5LUO6lY6hDT9ZK2l6ufm5KOKdrvE5K2SVpZzeuS9Iso7uWSzmokcY2UtLtcfd2dprh6SHpT0prou/hPVZRJe53VMq6015mkFpLekbQsimtKFWWS+z6aWbOcgFOBbwHzgcJqyqSAjcBJQC6wDOifcFwPApOj+cnAA9WU25uGOop9/8CtwC+j+UnAc40gpuuBhzPwmRoBnAWsrOb1i4FXAQFnA4saSVwjgVcyUF9dgbOi+TbAuir+lmmvs1rGlfY6i+ogP5rPARYBZ1cok9j3sdm2LMxsjZmtjSk2BNhgZpvM7ADwe2B8wqGNB56M5p8ELk14fzWpzfsvH+/zwAWSlOGYMsLM3gI+q6HIeOApCxYC7SV1bQRxZYSZfWJmS6L5PcAaoFuFYmmvs1rGlXZRHeyNFnOiqeJB58S+j802WdRSN+DjcsubSf5D08XMPoHwoQU6V1OuhaTFkhZKSiqh1Ob9Hy5jZiXAbqBjQvHUNiaAK6Jui+cl9UgwnrrIxOeptv4y6t54VdKAdO886i45k/DfcnkZrbMa4oIM1JmklKSlwDZgrplVW18N/X3MboiNNFaSioHjq3jpLjN7qTabqOK5oz59rKa46rCZnma2RdJJwBuSVpjZxqONrYLavP9E6qgGtdnfy8CzZva1pFsI/2mNSjCm2kp3XdXWEsKQD3slXQzMAPqma+eS8oEXgNvN7IuKL1exSlrqLCaujNSZmR0CBklqD0yXdJqZlT8WlVh9NelkYWZFR7mJzUD5/0q7A1uOcps1xiVpq6SuZvZJ1NzeVs02tkSPmyTNJ/z309DJojbvv6zMZknZQDuS7fKIjcnMdpZb/DXwQILx1EUin6ejVf6H0MxmS3pUUiczS3wMJEk5hB/kZ8zsxSqKZKTO4uLKZJ1F+9wVfe8vAsoni8S+j94NVbN3gb6STpSUSzhglNiZR5GZwHXR/HVApRaQpA6S8qL5TsA5wOoEYqnN+y8f75XAGxYdXUtIbEwV+rTHEfqcG4OZwLXRGT5nA7vLuhwzSdLxZf3akoYQfhd21rxWg+xXwOPAGjP7j2qKpb3OahNXJupMUkHUokBSS6AI+KBCseS+j+k8mt+YJuAyQhb+GtgKvBY9fwIwu1y5iwlnQ2wkdF8lHVdHYB6wPno8Lnq+EPhNND8MWEE4E2gFcGOC8VR6/8CPgHHRfAtgGrABeAc4KQ11FBfTT4BVUf28CZySps/Us8AnwMHos3UjcAtwS/S6gEeiuFdQzVl4GYjrtnL1tRAYlqa4ziV0kSwHlkbTxZmus1rGlfY6A84A3o/iWgncHT2flu+jX8HtnHMulndDOeeci+XJwjnnXCxPFs4552J5snDOORfLk4VzzrlYniycqwNJe+NL1bj+89FV90jKl/QrSRujUUTfkjRUUm4036QvmnXHFk8WzqVJNH5Qysw2RU/9hnB1bV8zG0AYLbeThQES5wETMxKoc1XwZOFcPURXFP9U0kpJKyRNjJ7PioZ+WCXpFUmzJV0ZrXY10RX5kvoAQ4EfmlkphKFbzGxWVHZGVN65RsGbuc7Vz+XAIGAg0Al4V9JbhKFXegOnE0YMXgM8Ea1zDuFqaoABwFILA8NVZSUwOJHInasHb1k4Vz/nEka2PWRmW4EFhB/3c4FpZlZqZp8Shhsp0xXYXpuNR0nkgKQ2DRy3c/XiycK5+qnuhjI13WhmH2HsHgjjCg2UVNN3MA/YX4/YnGtwniycq5+3gInRzWgKCLcufQd4m3DjpSxJXQi33yyzBjgZwMK9RxYDU8qNXtpX0vhoviOw3cwOpusNOVcTTxbO1c90wuify4A3gH+Jup1eIIzsuhL4FeEOa7ujdWZxZPK4iXATrA2SVhDuvVF2r4bzgdnJvgXnas9HnXWugUnKt3AHtY6E1sY5ZvZpdA+CN6Pl6g5sl23jReBOi79PvHNp4WdDOdfwXoluUpML3Be1ODCzfZLuIdwn+U/VrRzd1GmGJwrXmHjLwjnnXCw/ZuGccy6WJwvnnHOxPFk455yL5cnCOedcLE8WzjnnYnmycM45F+v/AQzLbImJ7R3NAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xae5e978>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot CV误差曲线\n",
    "test_means = grid.cv_results_['mean_test_score']\n",
    "test_stds = grid.cv_results_['std_test_score']\n",
    "train_means = grid.cv_results_['mean_train_score']\n",
    "train_stds = grid.cv_results_['std_train_score']\n",
    "\n",
    "# plot results\n",
    "n_Cs = len(Cs)\n",
    "number_penaltys = len(penaltys)\n",
    "test_scores = np.array(test_means).reshape(n_Cs, number_penaltys)\n",
    "train_scores = np.array(train_means).reshape(n_Cs, number_penaltys)\n",
    "test_stds = np.array(test_stds).reshape(n_Cs, number_penaltys)\n",
    "train_stds = np.array(train_stds).reshape(n_Cs, number_penaltys)\n",
    "\n",
    "x_axis = np.log10(Cs)\n",
    "for i, value in enumerate(penaltys):\n",
    "    plt.errorbar(x_axis, -test_scores[:,i], yerr = test_stds[:,i], label = penaltys[i] + 'Test')\n",
    "    plt.errorbar(x_axis, -train_scores[:,i], yerr = train_stds[:,i], label = penaltys[i] + 'Train')\n",
    "    \n",
    "plt.legend()\n",
    "plt.xlabel('log(C)')\n",
    "plt.ylabel('logloss')\n",
    "plt.savefig('LogisticGridSearchCV_C.png')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上图给出了L1正则和L2正则下、不同正则参数C对应的模型在训练集上及测试集上的logloss。可以看出在训练集上C越大（正则越少）的模型性能越好；但在测试集上当C=100时性能最好（L2正则）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 保存模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pickle\n",
    "pickle.dump(grid.best_estimator_, open('Diabetes_L1_org.pkl', 'wb'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
